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Description 

FIELD OF THE INVENTION 

The present invention is related to the data process- 
ing system based on the concept of neural network. 

BACKGROUND OF THE INVENTION 

The neural network in such kind of data processing 
system is constructed in layer state by settling parallelly 
neural cell model 1 in fig. 3 ("neuron", hereinafter). The 
neuron in each layer combines by synapses with all the 
neurons in adjacent other layer and inputs and outputs 
data. Concerning to neuron 1 , weights W1 , W2, W3, 
Wn are multiplied to data 11, 12, 13, In inputted from 
outside, and data O is outputted corresponding to the 
comparison between the sum of the multiplication and 
threshold 6. 

Various methods are possible for the comparison. 
When normalized function 1[f] is adopted, output data 
O is expressed as formula (1). 

0=1 [ZWn-ln-9] (1) 

When ZWn-ln is more than or equal to threshold 9, the 
neuron ignites and output data O is "1 ": when EWn-ln is 
less than threshold e, output data O is "0". 

Conventional neural network forms a neural layer 
by setting parallelly such neu rons and by connecting the 
neural layers in series. A conventional Neural Network 
is comprised of, for example, 3 layers of input layer, mid- 
dle layer and output layer as Perceptrons suggested by 
Rosenblatt: a neuron in each layer is connected with all 
neurons in adjacent layers by synapses. 

SUMMARY OF THE INVENTION 

In such a data processing system, the operation to 
adapt the weight of synapse of each neuron is called 
"learning". It is an important problem to guarantee its 
realization and to perform it efficiently. For example, in 
the back-propagation method, there are problems of 
getting out of a local minimum and of a convergence 
time. Especially the tendency is strong in neural network 
including a lot of middle layers. 

The present invention intends to solve the above 
problems of the prior art and has as an object to provide 
a learning method with the ability to execute efficient 
learning into the middle layer. 

The present invention relates to a learning method 
as set out in Claim 1 . 

BRIEF DESCRIPTION OF THE DRAWINGS 

Fig. 1 shows the approximate structure of character 
recognition system applied the first embodiment of the 
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present invention. 

Fig. 2 shows the outline structure of neuron in input 
layer, middle layer and output layer. 

Fig. 3 shows the outline structure of an embodiment 
5 of a neuron. 

Fig. 4 shows the diagram of the relationship be- 
tween the number of learning and weight. 

Figs. 5 (a), (b) and (c) show ignition pattern of each 
layer when neurons in a middle layer have the tendency 
10 of ignition in the case that the ignition rate of neuron in 
corresponding area is more than 15%. 

Figs. 6 (a), (b) and (c) show ignition pattern of each 
layer when neurons in a middle layer have the tendency 
of ignition in the case that the ignition rate of neuron in 
15 corresponding area is more than 20%. 

Figs. 7 (a), (b) and (c) show ignition pattern of each 
layer when neurons in a middle layer have the tendency 
of ignition in the case that the ignition rate of neuron in 
corresponding area is more than 25%. 
20 Figs. 8 (a), (b) and (c) show the neurons in input 
layer, middle layer and output layer in the case that the 
arrangement of neurons can be divided in honeycomb 
state. 

1 and N...neuron, 31. ..input layer, 32.. .middle layer, 
25 33.. .output layer. 

PREFERRED EMBODIMENT OF THE PRESENT 
INVENTION 

30 . Hereinafter, the present invention is explained with 
reference to the attached drawings. 

Fig. 1 shows a character recognition system com- 
prising data processing system related to the first em- 
bodiment. The character recognition system comprises 

35 video camera 10, preprocessing system 20, data 
processing system 30, post-processing system 40 and 
display 50. Video camera 10 is settled to input charac- 
ters/letters and connected to preprocessing system 20. 
Preprocessing system 20 is, for example, a well-known 

40 image processing system, which extracts the character- 
istics of inputted character/letter (the number of edge 
points or branch points, for instance) and outputs the 
data to data processing system 30. Data processing 
system 30, constructs a neural network as described lat- 

45 er, recognizes a character/letter based on the charac- 
teristics data inputted from preprocessing system 20, 
and outputs the data according to the recognition result 
to post-processing system 40. Recognition signal com- 
prises character codes for instance. Post-processing 

so . system 40 stores the output data as, for example, word 
processor data and simultaneously outputs to display 
50. Display 50 is constructed from CRT and indicates 
the characters/letters recognized by data processing 
system 40 on the display. 

ss The neural network used to construct data process- 
ing system 30 is constructed as a part of the hardware 
of a computer. Data processing system 30 is expressed 
in Fig. 1 as a model. It comprises input layer 31 , middle 
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layer 32, and output layer 33: middle layer 32 is settle 
between input layer 31 and output layer 33. In the 
present embodiment, each layer 31 , 32, and 33 com- 
prises the same number of neurons of N: neuron in input 
layer 31 are connected to all the neurons comprised in 5 
middle layer 32, and those in middle layer 32 are con- 
nected to all the neurons comprised in output layer 33. 

As mentioned above with referring to Fig. 3, each 
neuron N outputs the data of "1 " or "0" according to the 
normalized function in formula (1). Neuron N is con- io 
structed by means of operational amplifiers, for exam- 
ple. Weight Wn to be multiplied to the data to be inputted 
to each neuron N is obtained by means of a changeable 
resister connected to input terminal. Threshold function 
is realized by means of switching elements. That is, '5 
learning is executed amending output data and chang- 
ing weight Wn by means of a changeable resister ac- 
cording to output data of each neuron. 

Fig. 2 shows input layer 31 , middle layer 32 and out- 
put layer 33 as a model: each layer 31 , 32 and 33 com- 20 
prises the same number of neurons. Here, for a simple 
explanation, the number of neurons is 36 in each layer 
31, 32 and 33: neurons are arranged 6 in lateral and 
longitudinal directions in each layer. It is presumed that 
the point in left below edge as the origin, and that the 25 
neuron in the location of i-th left and j-th upper as Nij. 

Each neuron in input layer 31 ignites by character- 
istics data obtained through video camera 10. When ig- 
nition combination of N11 , N1 2, N21 and N22 expresses 
the number of edge points, and N1 3, N14, N23and N24 30 
expresses the number of branch point, the ignition pat- 
tern of neurons in input layer 31 is decided artificially 
according to characters/letters to be inputted. 

On the other hand in output layer 33, a character/ 
letter is expressed by neuron Nii on diagonal line from 35 
the edge on lower left to the edge on upper right. That 
is, the neurons on the diagonal line express the charac- 
ter code of the character/letter, and the ignition pattern 
of neuron in output layer 33 is decided artificially. In the 
present embodiment, the kind of ignition pattern is 64 40 
for neuron Nii on diagonal line. Therefore, it is possible 
to recognize 64 kinds of characters/letters: recognition 
of alphabets is possible, for example. 

Before executing learning of character/letter recog- 
nition, neurons in output layer 33 do not ignite even if « 
character/letter data is inputted to data processing sys- 
tem 30. This neuron can lear by igniting: its learning is 
concluded when a certain ignition pattern is appeared 
according to inputted character/letter data. Input and 
output pattern for learning is the representative data to so 
be processed in neural network: the data to be actually 
processed exist in a distributed form. Learning has been 
executed until association for the representative input 
and output data is performed adequately. When learning 
is concluded, input layer 31 and output layer 33 show ss 
ignition pattern decided artificially according to inputted 
character/letter data as mentioned above. It is pre- 
sumed that ignition pattern in each layer of input layer 



31, middle layer 32 and output layer 33 changes 
smoothly in these layers of 31 , 32 and 33. In the present 
embodiment, the weights in middle layer 32 and output 
layer 33 are changed in order to ignition pattern in these 
layers change smoothly on learning process. 

First, it is decided if neuron Nij in middle layer 32 
ignites or not considering the ignition of the neuron ad- 
jacent to each neuron Nij in input layer 31 and output 
layer 33 corresponding to the neuron Nij. That is, when 
neurons ignite or are to be ignited with more than a cer- 
tain rate among corresponding neuron Nij and neurons 
around it in input layer 31 and output layer 33, neuron 
Nij in middle layer 32 is judged to have the tendency of 
ignition. Weights of designated neurons in middle layer 
32 and output layer 33 are increased in order to obtain 
such ignition. 

The judging method if neurons in middle layer are 
to ignite or not is described referring to Fig. 5 (a), (b) and 
(c). Fig. 5 (a) shows the ignition pattern in input layer 
31, Fig. 5 (b) shows that in middle layer 32, Fig. 5 (c) 
shows that in output layer 33: in these figures, black cir- 
cles show ignited neurons, white circles show united 
neurons. In this embodiment, input layer 31 shows the 
ignition pattern as in the diagram according to charac- 
teristics data of inputted character/letter: the ignition 
pattern in output layer 33 have to be what is shown in 
the diagram so as to recognize it. That is, the ignition 
pattern in input layer 31 in the diagram is fixed according 
to input character/letter: the ignition pattern in output 
layer 33 is what on the completion of learning. The igni- 
tion pattern in middle layer 32 is the one expected by 
the method of the present embodiment. 

In Fig. 5 (a), (b) and (c), first, it is examined the ig- 
nition condition of, for instance, neurons N24 and 8 of 
neurons adjacent to it as to neuron N24 in input layer 
31 corresponding to neuron N24 in middle layer 32. That 
is, examining ignition condition of neurons in the area 
surrounded by broken line A, 3 neurons ignite in it. In 
the same way, examining the ignition distribution of neu- 
ron N24 in output layer 33 corresponding to N24 in mid- 
dle layer 32 and neurons adjacent to it - that is, the area 
' surrounded by broken line B - one neuron ignites. There- 
fore, 4 neurons ignite among 18 neurons in input layer 

31 and output layer 33 corresponding to neuron N24 in 
middle layer 32 and adjacent neurons to it. Neuron N24 
in middle layer 32 is judged to have the tendency of ig- 
nition when it is decided that the neuron in middle layer 

32 is to ignite in such a case of more than or equal to 
15% of neurons ignite among the neurons in areas A 
and B. As for Fig. 5 (a), (b) and (c), it is decided that a 
neuron in middle layer 32 has the tendency of ignition 
when more than or equal to 15% of neurons ignite 
among the neurons in input layer 32 and output layer 33 
corresponding to the neuron in middle layer 32. 

Concerning to the neurons on an edge, the ignition 
condition of 6 neurons shown with broken line C is ex- 
amined: concerning to the neurons on the corner, that 
of 4 neurons shown with broken line D is examined. 
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In Fig. 5 (a), (b) and (c), 30 of black neurons are 
judged to have the tendency of ignition among the neu- 
rons in middle layer 32. On the character/letter recogni- 
tion learning, only certain values are increased in order 
for the weight of each neuron in middle layer 32 andout- 
put'layer 33 to obtain the ignition pattern obtained in this 

The increase of weight will be explained using Fig. 
3, when output data O of neuron 1 in middle layer is the 
same value on ignition (for instance "1"), the weight of 
synapse ("W2" and "W3" in this case) corresponding to 
the data inputted from the ignited neuron (for instance 
"12" and "13") is, for example, increased 5% among the 
neurons in input layer connected to neuron 1. The syn- 
apse weight of a neuron in output layer is processed in 
the same way: as described above, the weight of syn- 
apse corresponding to the neuron in middle layer decid- 
ed to be ignited is increased 5%, for example. 

The weight of middle layer 32 and output layer 33 
is increased in order for ignition pattern to be changed 
smoothly among each layer of input layer 31 , middle lay- 
er 32 and output layer 33. Here, after deciding whether 
each neuron in input layer 31 and output layer 33 is to 
ignite on the same frequency as much as possible for 
all input characters/letters, it is possible for each neuron 
in middle layer 32 to ignite at the some time. In this way 
it is possible to prevent to fall in local minimum-by it, and 
each neuron in middle layer 31 ignites at the some time. 
It means that it is possible to avoid the generation of 
neurons with.no ignition and to work neurons in middle 
layer 32 efficiently. 

The increase of synapse weight on one learning 
changes as in Fig. 4 according to the number of learning. 
The learning of a whole system is executed gradually 
by plural learning, and simultaneously, fine adjustment 
is carried out for fine change. The learning speed is de- 
veloped by rapid increase of weight in the beginning. 

■ In the embodiment above, middle layer 32 is con- 
nected to input layer 31 and output layer 33, that is, the 
layers are 3. However, the present invention does not 
set limits on embodiments as 3 layers of neural net- 
works: it is possible to apply it for neural network with 
more than 4 layers. In this case, selecting thefirst middle 
layer before everything, the weight of synapse in the first 
middle layer and output layer is increased in order that 
each neuron in the middle layer has a tendency to ignite 
when the neurons in input layer and output layer corre- 
sponding each other and neurons adjacent to them ig- 
nite with more than a certain rate. When learning is con- 
cluded so far as a certain step concerning to all the input 
characters/letters, adding the second middle layer new- 
ly, the weight of synapse is increased in the second mid- 
dle layer and the layer connected to output side of it (out- 
put layer or the first middle layer), considering whether 
corresponding neurons and adjacent to it are to ignite 
or not with more than a certain rate, in the same way as 
to the second middle layer. Consequently, weight distri- 
bution in the case with 4 layers can be obtained. As to 



the case with more than 5 layers, the same process is 
performed. 

Figs. 6 (a), (b) and (c) show ignition patterns in lay- 
ers 31, 32 and 33 on recognizing the same character/ 

s letter as in Figs. 5 (a) to (c): Figs. 6 (a) and (c) are quite 
the same as Figs. 5 (a) and (c). The ignition pattern in 
middle layer 32 is decided to have the tendency of igni- 
tion when the neurons corresponding to those in input 
layer 31 , output layer 33 and adjacent to them ignite 

10 more than 20%, different from Fig. 5 (b). 

Figs. 7 (a), (b) and (c) also show ignition patterns 
in each layer on recognizing the same character/letter 
as in Figs. 5 (a) to (c): Figs. 6 (a) and (c) are quite the 
same as Figs. 5 (a) and (c). In Figs. 7 (a) to (c), the ig- 

is nition pattern in middle layer 32 is decided to have the 
tendency to ignite when neurons corresponding to those 
in input layer 31 , output layer 33 and adjacent to them 
ignite more than 25%. 

As understood easily from the comparison with Fig. 

20 5 (b), Fig. 6 (b) and Fig. 7 (b), the ignition pattern in mid- 
dle layer 32 has the tendency with the characteristics of 
ignition pattern in both input layer and output layer. The 
higher the ignition rate of a neuron in the area corre- 
sponding to input layer 31 and output layer 33 is the 

2S smaller is the number of neurons that ignite in middle 
layer 32. ■ 

Though the areas A, B, C and D to decide neurons 
adjacent to each neuron are rectangles in embodiments 
above, a form of an area is not set a limit to it. When the 

so arrangement of neurons can be divided in the state of 
honeycomb, the form of the area is hexagon as shown 
in Figs. B (a), (b) and (c). In this case, area E in input 
Iayer31 and area Fin output layer corresponding to neu- 
ron N in middle layer 32 are comprised of 7 neurons. 

3S Neuron N in middle layer 32 has the tendency to ignite 
when 14 neurons in area E and F ignite with more than 
a certain rate. 

The input and output data for a neuron is not set a 
limit to biharized one: it may be multi-valued and analog 

40 data when the data is digital. 

An output layer may be constructed to define a char- 
acter/letter for recognition by all neurons: it is not nec- 
essary to be constructed to express character codes by 
neurons on diagonal line. 

45 . The number of neurons in the layers of 31, 32 and 
33 is not set a limit to 36: it is possible to settle it accord- 
ing to the number of kinds of characters/letters to be rec- 
ognized. 

The present invention can be applied to not only 
50 character/letter recognition but also to pattern recogni- 
tion and voice recognition. 

As mentioned above, it is possible to execute effec- 
tive learning on middle layer of the neural network of the 
present invention. 

ss 
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A learning method for transforming an untrained 
network to a trained network, said network compris- 
ing an input layer (31 ), a middle layer (32), and an 5 
output layer (33), each said layer including a plural- 
ity of neurons, each said neuron being a signal 
processing element, said middle layer being ar- 
ranged between said input and output layers where- 
in each neuron in said middle layer is connected to 10 
each neuron in said input and output layers, said 
method comprising the steps of: 

determining an activation pattern of said neu- 
rons in said input layer; « 

determining an activation pattern of said neu- 
rons in said output layer; 

defining two neighborhoods of said neurons 20 
' one in each said input and output layers corre- 
sponding to a neuron in said middle layer; 

determining a rate of activation in said neigh- 
borhood of neurons, said rate being the total 25 
number of neurons becoming active in said 
neighborhood in said input and output layers di- 
vided by the total number of neurons in said 
neighborhood in said input and output layers; 

30 

comparing said rate to a threshold value; 

determining that said neuron in said middle lay- 
er has a tendency to become active when said 
rate is greater than said threshold value; 35 

if said neuron in said middle layer has a ten- 
dency to become active, increasing the value 
of weights applied to all neurons in said input 
layer which are connected to said neuron in 40 
said middle layer and weights applied to all neu- 
rons in said output layer which are connected 
to said neuron in said middle layer, in order to 
increase the likelihood of activation of said neu- 
ron in said middle layer; and 45 

repeating the above steps for each said neuron 
in said middle layer, thereby generating a de- 
sired activation pattern of neurons in said mid- 
dle layer corresponding to said trained network, so 

The learning method of claim 1 further including the 
steps of: 

a. adding an additional middle layer between 55 
said input layer and said middle layer or be- 
tween said middle layer and said output layer, 
said additional middle layer having a plurality 



b. connecting each neuron in said additional 
middle layer to each neuron in the two layers 
adjacent to said additional middle layer; 

c. defining two neighborhoods of neurons one 
in each said adjacent layers corresponding to 
a neuron in said additional middle layer; 

d. determining a rate of activation in said neigh- 
borhood of neurons, said rate being the total 
number of neurons becoming active in said 
neighborhood, divided by the total number of 
neurons in said neighborhood; 



3. comparing s 



d rate to a threshold value; 



f. if said rate is greater than the threshold value, 
increasing the value of weights applied to all 
neurons in one of said adjacent layers which 
are connected to said neuron in said additional 
middle layer and weights applied to all neurons 
in said other adjacent layer which are connect- 
ed to said neuron in said additional middle lay- 
er, in order to increase the likelihood of of said 
neuron being active in said additional middle 
layer, and 

g. repeating steps c.-f. for each neuron in said 
additional middle layer, thereby generating a 
desired pattern of neurons being active in said 
additional middle layer corresponding to said 
trained network. 

The learning method of claim 1 wherein said neigh- 
borhood includes a neuron in said input layer and a 
neuron in said output. layer corresponding to the 
same position as said neuron in said middle layer 
and all neurons adjacent to said neuron in said input 
layer and said output layer. 

The learning method of claim 1 , wherein said learn- 
ing method is a method for recognizing characters, 
said pattern of neurons being active in said input 
layer corresponds to characteristics data of an im- 
age, and said pattern of neurons being active in said 
output layer corresponds to character data of said 
image. 



Patentanspruche 

1. Lernverfahren zum Umwandeln eines untrainierten 
Netzwerkes in ein trainiertes Netzwerk, wobei das 
Netzwerk eine Eingangsschicht (31), eine Mittel- 
schicht (32) und eine Ausgangsschicht (33) auf- 
weist, von denen jede eine Vielzahl von Neuronen 
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enthalt und jedes Neuron ein Signalverarbeitungs- 
element ist, wobei weiterhin die mittlere Schicht so 
angeordnet ist, da3 sie zwischen der Eingangs- 
schicht und der Ausgangsschicht liegt, derart, daB 
jedes Neuron der mittleren Schicht mit jedem Neu- s 
ron der Eingangs- und Ausgangsschicht verbunden 
ist und wobei das Verfahren folgende Schritte auf- 
weist: 

Bestimmen eines Aktivierungsmusters der w 
Neuronen in der Eingangsschicht; 

Bestimmen eines Aktivierungsmusters der 
Neuronen in der Ausgangsschicht; 

75 

Definieren von zwei Nachbarumgebungen fur 
die Neuronen in der Eingangsschicht und der 
Ausgangsschicht entsprechend einem Neuron 
der mittleren Schicht; 

20 

Bestimmen einer Aktivierungsrate innerhalb 
der Nachbarumgebung der Neuronen, wobei 
die Rate der gesamten Anzahl von Neuronen, 
die in der Nachbarumgebung innerhalb der Ein- 
gangsschicht und der Ausgangsschicht aktiv & 
sind, dividiert durch die Gesamtanzahl der 
Neuronen in der Nachbarschaft der Eingangs- 
schicht und der Ausgangsschicht entspricht; 

Vergleichen der Rate mit einem Schwellwert; 30 

Bestimmen, da3 das Neuron der mittleren 
Schicht eine Aktivierungstendenz aufweist, 
wenn die Rate groBer ist als der Schwellwert; 

35 

fur den Fall, daft das Neuron in der mittleren 
Schicht eine Aktivierungstendenz aufweist, Er- 
hdhen der Gewichtungswerte der Neuronen in 
der Eingangsschicht, die mit dem Neuron in der 
mittleren Schicht verbunden sind und der Neu- 40 
ronen in der Ausgangsschicht, die mit dem 
Neuron in der Mittelschicht verbunden sind, urn 
die Wahrscheinlichkeit einer Aktivierung des 
Neurons in der mittleren Schicht zu erhohen; 
und 45 

Wiederholen derobigen Schritte fur jedes Neu- 
ron in der mittleren Schicht, wodurch ein ge- 
wunschtes Aktivierungsmuster von Neuronen 
in der mittleren Schicht entsprechend einem so 
trainierten Netzwerk erzeugt wird. 

Lemverfahren nach Anspruch 1, weiterhin aufwei- 
send die Schritte: 

55 

a) Addieren einer zusatzlichen Mittelschicht 
zwischen der Eingangsschicht und der Mittel- 
schicht oder zwischen der Mittelschicht und der 



Ausgangsschicht, wobei die zusatzliche Mittel- 
schicht eine Vielzahl von Neuronen aufweist; 

b) Verbinden jedes Neurons in der zusatzlichen 
Mittelschicht mit jedem Neuron in den zwei an- 
grenzenden Schichten zu der zusatzlichen Mit- 
telschicht; 

c) Definieren von zwei Nachbarumgebungen 
von Neuronen in den beiden angrenzenden 
Schichten der zusatzlichen Mittelschicht; 

d) Bestimmen einer Aktivierungsrate in der 
Nachbarschaftsumgebung, wobei die Rate der 
gesamten Anzahl von aktiven Neuronen in der 
Nachbarschaft, dividiert durch die gesamte An- 
zahl. von Neuronen in der Nachbarschaft ent- 
spricht; 

e) Vergleichen der Rate mit einem Schwellwert; 

f) fur den Fall, daB die Rate groBer ist als der 
Schwellwert, Erhohen der Gewichtungswerte, 
die den Neuronen in der einen angrenzenden 
Schicht zur zusatzlichen Mittelschicht zugefugt 
werden und der Gewichte, die alien Neuronen 
in der anderen angrenzenden Schicht zugefugt 
werden, die mit dem Neuron in der zusatzlichen 
Mittelschicht verbunden sind, urn die Wahr- 
scheinlichkeit zu erhohen, daB Neuronen in der 
zusatzlichen Mittelschicht aktiviert werden, und 

g) Wiederholen der Schritte c) bis f) fur jedes 
Neuron in der zusatzlichen mittleren Schicht, 
wodurch ein gewunschtes Aktivierungsmuster 
fur Neuronen in der zusatzlichen Mittelschicht 
erzeugt wird entsprechend einem trainierten 
Netzwerk. 

3. Lemverfahren nach Anspruch 1, wobei die Nach- 
barschaft ein Neuron in der Eingangsschicht' und 
ein Neuron in der Ausgangsschicht entsprechend 
der Position des Neurons in der Mittelschicht auf- 
weist sowie alls Neuronen, die benachbart zu dem 
entsprechenden Neuron in der Eingangsschicht 
und der Ausgangsschicht sind. 

a Lemverfahren nach Anspruch 1, wobei das Lern- 
verfahen ein Verfahren zum Erkennen von Zeichen 
ist, wobei das Aktivierungsmuster der Eingangs- 
schicht charakteristischen Daten eines Bildes ent- 
spricht und das Aktivierungsmuster in der Aus- 
gangsschicht Zeichendaten des Bildes entspricht. 



Revendications 

1. Methode d'apprentissage pour transformer un re- 



11 EP 0 450 522 B1 12 



seau non forme en un reseau forme, ledit reseau 
comprenant une couche d'entree (31), une couche 
centrals (32), et une couche de sortie (33), chaque 
dite couche incluant une pluralite de neurones, cha- 
que dit neurone etant un element de traitement de s 
signaux, ladite couche centrale etant disposee en- 
tre lesdites couches d'entree et de sortie, ou chaque 
neurone dans ladite couche centrale est connecte 
a chaque neurone dans lesdites couches d'entree 
et de sortie, ladite methode comprenant les etapes io 
consistant a : 

determiner une configuration de mise a I'etat 
actif desdits neurones dans ladite couche 
d'entree ; 75 
determiner une configuration de mise a I'etat 
actif desdits neurones dans ladite couche de 
sortie ; ; • 

definir deux voisinages desdits neurones dont 
un dans chacune desdites couches d'entree et 20 
de sortie correspondent a un neurone dans la- 
dite couche centrale ; 

determiner un taux de mise a I'etat actif dans 
ledit voisinage des neurones, ledit taux etant le 
nombre total de neurones devenant actifs dans 25 
ledit voisinage desdites couches d'entree et de 
sortie, divise par le nombre total de neurones 
dans ledit voisinage desdites couches d'entree 
et de sortie ; 

comparer ledit taux a une valeur de seuil ; so 
determiner que ledit neurone dans ladite cou- 
che centrale a tendance a devenir actif quand 
ledit taux est superieur a ladite valeur de seuil ; 
si ledit neurone dans ladite couche centrale a 
tendance a devenir actif, augmenter la valeur 35 
des poids appliques a tous les neurones dans 
ladite couche d'entree qui sont connectes audit 
neurone dans ladite couche centrale et des 
poids appliques atous les neurones dans ladite 
couche de sortie qui sont connectes audit neu- 40 
rone dans ladite couche centrale, afin d'aug- 
menter la probabilite de mise a I'etat actif dudit 
neurone dans ladite couche centrale ; et, 
repeter les etapes ci-dessus pour chaque dit 
neurone dans ladite couche centrale, en gene- 45 
rant ainsi une configuration de mise a I'etat actif 
voulue des neurones dans ladite couche cen- 
trale correspondant audit reseau forme. 



b. connecter chaque neurone dans ladite cou- 
che centrale supplementaire a chaque neurone 
dans les deux couches adjacentes a ladite cou- 
che centrale supplementaire ; 

c. definir deux voisinages des neurones dont 
un dans chacune desdites couches adjacentes 
correspondant a un neurone dans ladite cou- 
che centrale supplementaire ; 

d. determiner un taux de mise a I'etat actif dans 
ledit voisinage des neurones, ledit taux etant le 
nombre total de neurones devenant actifs dans 
ledit voisinage, divise par le nombre total de 
neurones dans ledit voisinage ; 

e. comparer ledit taux a une valeur de seuil ; 

f . si ledit taux est superieur a la valeur de seuil, 
augmenter la valeur des poids appliques a tous 
les neurones dans une desdites couches adja- 
centes qui sont connectes audit neurone dans 
ladite couche centrale supplementaire et des 
poids appliques a tous les neurones dans ladite 
autre couche adjacente qui sont connectes 
audit neurone dans ladite couche centrale sup- 
plementaire, afin d'augmenter la probabilite 
que ledit neurone soit actif dans ladite couche 
centrale supplementaire ; et, 

g. repeter les etapes c.-f. pour chaque neurone 
dans ladite couche centrale supplementaire, en 
generant ainsi une configuration voulue de 
neurones actifs dans ladite couche centrale 
supplementaire correspondant audit reseau 
forme. 

3. Methode d'apprentissage selon la revendication 1 , 
dans laquelle ledit voisinage comprend un neurone 
dans ladite couche d'entree et un neurone dans la- 
dite couche de sortie correspondant a la meme po- 
sition que ledit neurone dans ladite couche centrale 
et tous les neurones adjacents audit neurone dans 
ladite couche d'entree et ladite couche de sortie. 

4. Methode d'apprentissage selon la revendication 1 , 
dans laquelle ladite methode d'apprentissage est 
une methode pour reconnaftre des caracteres, la- 
dite configuration des neurones actifs dans ladite 
couche d'entree correspond aux donnees caracte- 
ristiques d'une image, et ladite configuration des 
neurones actifs dans ladite couche de sortie corres- 
pond a des donnees de caractere de ladite image. 



2. Methode d'apprentissage selon la revendication 1 , 
incluant en outre les etapes consistant a : 

a. ajouter une couche centrale supplementaire 
entre ladite couche d'entree et ladite couche 
centrale ou entre ladite couche centrale et la- 
dite couche de sortie, ladite couche centrale 
supplementaire comportant une pluralite de 
neurones ; 
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Fig. 5 
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Fig. 7 
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